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Abstract 


School closures are increasingly common among U.S. public schools, driven by both budgetary constraints 
and accountability pressures to turnaround low-performing schools. This paper contributes to the nascent 
literature on school closures by evaluating student achievement and mobility outcomes in a large-scale 
restructuring effort in Washington, D.C. in which 32 elementary and middle school campuses were closed or 
consolidated in the summer of 2008. Using longitudinal data, we investigate how student outcomes change 
in relation to this initiative with an instrumental variables strategy that counters the endogeneity of student 
assignment across schools before and after the restructuring occurred. The results show that the academic 
performance of students directly affected by the school restructuring experienced a temporary decline, but it 
rebounded by the second school year after restructuring occurred. Additionally, we find no evidence of 


closure adversely inducing further mobility among affected students. 


Introduction 


Closing schools is an increasingly common strategy among public school districts to promote 
both efficiency and performance. Faced with tighter budget constraints, declining enrollments, and 
performance pressure, many districts are closing or consolidating schools in order to reallocate 
increasingly scarce resources from facilities to instruction. Apart from efficiency concerns, federal 
policymakers are promoting school closures as a way to better student performance. Initially introduced 
as a sanction for enforcing school accountability under the No Child Left Behind Act (NCLB), the U.S. 
Department of Education’s Race to the Top initiative (RttT) and School Improvement Grant program 
(SIG) are promoting school closures as a key strategy for turning around chronically low-performing 
schools—either through converting the school to a charter school or by closing the school altogether 
and enrolling the students in neighboring schools likely to provide better opportunities. While the 
intentions behind school closures are clear, the effects on students are not. Prior research suggests the 
forced movement of students may adversely affect student learning; and, by dislocating students, school 
closure may promote even further mobility. 

This study investigates how student outcomes, namely student achievement and mobility, were 
affected in relation to a policy decision that closed or consolidated 32 elementary and middle schools in 
the summer of 2008 in the District of Columbia Public Schools system (DCPS) in Washington, D.C. 
Longitudinal student data spanning the periods before and after the school restructuring effort enables 
the estimation of the restructuring effect on student achievement and various forms of student 
mobility. An instrumental variables strategy is used to counter the endogeneity of school assignment 
prior to and after closure. The resulting estimates of the investigation show statistically significant, albeit 
temporary, declines in student achievement in the year of the closure announcement and in the first 
year following the closures. However, no detectable differences in student achievement are observed in 


the second year after closures. Student performance drops in affected students by 0.10 to 0.20 


standard deviations in the very near term, but appears to rebound very quickly and is indistinguishable 
from students unaffected by the restructuring initiative. The results also show no evidence of student 


mobility increasing in relation to the large-scale restructuring effort. 


National and Local Context 


Nationwide, three different pressures threaten to close public schools. The first comes from 
declining enrollments, which is especially common in urban centers in the Midwest and northeastern 
United States. A recent report shows enrollment declines over the past decade of 17 percent or more in 
districts located in seven major cities in these regions (Pew Charitable Trusts, 2011).’ While enrollment 
declines reflect long-term trends, the recent financial crisis appears to have anecdotally accelerated the 
number of closures as districts attempt to reallocate resources from facilities to instruction in an effort 
to dampen the effect of general budget cuts in education (Johnson, 2010).” The second threat comes 
from the rapid expansion of charter schools in recent years, particularly in urban areas (Lake, 2010), and 
co-incidental overall declines in public school enrollment.’ Finally, accountability pressures originating 
from NCLB threaten to close chronically low-performing schools that fail to meet adequate yearly 
progress goals (AYP); and this closure threat still persists as part of the Department of Education’s 
prescribed turnaround models under the RttT and SIG programs. Secretary of Education Arne Duncan 
estimated 82 percent of the nation’s schools would fail to meet AYP in the 2010-11 school year, 


therefore initiating sanctions that could ultimately include school closure (barring a waiver for the 


' School closures are commonly associated with urban school districts, yet this is also a persistent issue in rural 
school districts as well (Bard et al., 2006). 

° Though academic research cataloging recent closures is limited, the topic has received considerable coverage in the 
news media, including Khan (2010) and Leslie (2011). 

> The relationship between the expansion of the charter movement and the decline of urban school districts has been 
sufficiently strong to prompt the NAACP and the United Federation of teachers to sue the New York City 
Department of Education in a recent, albeit unsuccessful, lawsuit (Snow, 2011). 
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legally mandated interventions).’ Hence, both local and national pressures combine to threaten school 
closures. 

The District of Columbia Public Schools system is a poster-child district for these conditions. In 
the ten years leading up to the 2006-07 school year, total student enrollment in DCPS decreased by over 
20,000 students (26 percent). Meanwhile, total student enrollment in D.C.’s charter schools (first 
opened in the 1998-99 school year) increased by a similar amount.° DCPS ranked near the top among 
urban districts in facility-related per-pupil spending and near the bottom in instructional per-pupil 
spending as a fraction of total budget in 2004-05.° Additionally, DCPS ranked at the bottom in math 
performance in grades 4 and 8 against all participating districts in the 2007 Trial Urban District 
Assessment of the NAEP (Lutkus, et al., 2007). The inefficient use of resources, substandard school 
facilities, and consistent low performance were three of the key issues that Michelle Rhee faced when 


she was appointed as Chancellor of Schools for DCPS in June of 2007 (Nakamura, 2007). 


The DCPS Closure Initiative 


Prior to Chancellor Rhee’s arrival in the district, a 2006 Parthenon Group report commissioned 
by the district found over two-thirds of DCPS buildings received ratings of “poor or unsatisfactory” and 
over a third of them were at less than 65% capacity for the 2005-06 school year. The DCPS Master 
Facilities Plan, issued in the spring of 2006, contained unprecedented directives to close and consolidate 
a number of DCPS campuses, which were intended to improve administrative efficiency. When 
Chancellor Rhee took office in 2007, she took control of the process already underway to identify 


schools for the necessary restructuring as outlined in the DCPS Master Facilities Plan. 


* «Duncan Says 82 Percent of America's Schools Could ‘Fail’ Under NCLB This Year.” March 9, 2011 DOE press 
release retrieved from http://www.ed.gov. 

> Charter school enrollment gains have roughly offset the loss of DCPS students; total combined enrollment between 
DCPS and public charter schools has been generally steady during this period. Seventy new charter campuses 
opened during this period. “Reorganizing DCPS Schools to Support Comprehensive Educational Programs and 
Services” November 2007. 21* Century School Fund, Brookings Institute, DC Government, Urban Institute. 

° Figures exclude special education expenses (Keating and Haynes, 2007). 
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Candidate schools for closure or consolidation were selected on the basis of low and declining 
enrollment: those exhibiting a five-year decline in enrollment greater than the district median and a 
2006 enrollment below the district median. Forty schools met these primary criteria. Following this 
primary identification of 40 schools, DCPS further vetted the list to account for other factors that might 
make campuses more or less suitable for closure. The main criteria under consideration during this part 
of the process included projected future neighborhood population trends, building conditions, the 
feasibility of relocating students to neighboring schools, etc.’ Between December 2007 and January 
2008, DCPS engaged in a series of community dialogues and meetings in which community members 
were given the opportunity to express their concerns over the proposed closures. The final restructuring 
decisions came out of this process, and on February 1, 2008, DCPS announced a list of 23 schools 
scheduled for closure pending a final community hearing process (some of which were to be 
consolidated), which was finalized in the spring of 2008.° Further detail on the restructured schools will 
be provided in the Data section. 

Although the closure initiative in DCPS was driven primarily by enrollment figures (determined 
by local pressures), the outcomes of this restructuring process also have implications for federal policies 
that promote school closure as a remedy for poor performance. Note that consistent low performance 
in a school was not an explicit criterion in the process for identifying which DCPS schools to close, yet it 
was correlated with these indicators.” For instance, as a result of consistently failing to meet AYP 


standards, 18 DCPS elementary or middle schools received mandates to restructure in 2008-09, of which 


7 “Reorganizing DCPS Schools to Support Comprehensive Educational Programs and Services” November 2007. 
21" Century School Fund, Brookings Institute, DC Government, Urban Institute. 

* “Fenty, Rhee and Reinoso Announce Revised School Consolidation Plan.” February 1, 2008 DCPS press release 
retrieved from http://newsroom.dc.gov. 

* One may reasonably expect a positive correlation between student enrollment and school quality: if neighborhood 
parents are sensitive to school quality, then their demand for schooling (i.e., a school’s enrollment) will be 
responsive to differences in quality. Hence, low and declining enrollments are expected to be correlated with low 
school quality. 


9 met the primary criterion of below-median declines and below-median enrollment.”° Moreover, in 
choosing whether to close or consolidate schools on a school-by-school basis, a preference for 
minimizing disruption among schools that were making progress was communicated in DCPS documents 


describing the decision-making rationale.*”” 


Because many of the schools targeted for intervention in 
DCPS were among some of the district’s lowest performing schools, we argue the findings here are 


relevant to understanding how accountability-induced closures may affect students. 


Research on School Closure 


School closure appears to have clear cost-saving benefits, but its use as a policy tool for 
improving student performance is not clear. NCLB first introduced the threat of school closure on the 
premise of inducing administrative and instructional staff to improve student performance in low- 
performing schools. Under current prescriptions on turning around the nation’s lowest performing 
schools, a school may be converted to a charter school or closed down entirely; yet, improved student 
performance is not necessarily a natural outcome of such a process. School closure may influence 
student outcomes in three ways: as a threat (inducing behavioral responses in threatened schools), in its 
actual application (forcing moves, presumably to a better school), and through indirect means (parent or 
student responses to closure). The extant research documenting how closure affects students is 
somewhat limited; an overview of this evidence is presented below. 

First, the threat of closure may induce school staff to exert greater levels of effort to avoid 
potential closure, particularly in cases where the closure decision is explicitly based on low performance 
measures. Rouse, et al. (2007) find evidence of such threat-induced responses in Florida. Yet, inasmuch 


as the closure decision may be marginally affected by school level efforts that are not productive to 


Th total, 27 schools received mandates to restructure, which includes high schools. Counts based on the school 
listing provided at http://dcist.com/2008/05/15/fenty_and_rhee_1.php. 

'' Final Proposal: A Revised Plan, Powerpoint presentation available at http://dc.gov/DC/ accessed on 11/25/2009. 
'? As a result, students in consolidated (but not closed) schools generally outperformed those in neighboring schools 
that were closed. This is discussed further in the Data section. 
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students’ educational outcomes (e.g., teaching to the test), this threat-induced effort may not 
necessarily be productive for students (see a review in Rice and Malen, 2010). 

Second, when a school is actually closed, student dislocation alone may have a detrimental 
effect on students—they must develop relationships with a new group of teachers and peers, cope with 
a different school culture, and adapt to any differences in curriculum or instruction. Conversely, the 
move could improve student learning if the instructional quality of the new school is sufficiently higher 
than that of the closed school, overcoming any adverse effects of being forced to move. Though the 
empirical evidence on incidental student mobility provides a clear consensus of adverse effects on 
achievement (e.g. Hanushek et al., 2004, and Xu et al., 2009), the forced mobility due to school closures 
does not appear to have the same consensus. Analyses from de la Torre and Gwynne (2009) fail to find 
detectable changes in student achievement in the years following school closure but Engberg et al. 
(2011) do find a significant negative effect in the first one to two years following closure. In addition, 
both of these analyses find students who were transferred to higher-growth schools (relative to the 
closed school) did experience a significant gain in achievement, all things equal. On the contrary, 
evidence from Kirshner et al. (2010) shows significant declines in student achievement following the 
announcement and closure of a high school in addition to a temporary increase in the dropout rate. 
Accompanying qualitative data documented common themes among dislocated students such as 
strained relationships with other peers and less assistance from teachers and staff at the new school. 

And third, school closure may indirectly affect school-related behaviors in students by forcing 
family adjustments. For instance, higher transportation costs may reduce school attendance for 
constrained students, or parents may take the opportunity to exercise school choice explicitly (by 
moving to a charter school) or implicitly (by moving residences). The shuffling of students across schools 
could plausibly frustrate parents and induce them to move out of the area entirely, and attend schools 


in a neighboring jurisdiction. Only two studies have previously addressed indirect outcomes of closure. 


First, Engberg et al. (2011) shows a small temporary increase in absences among dislocated students in 
the year immediately following school closure.** Second, de la Torre and Gwynne (2009) document 
higher rates of subsequent school mobility among students forced to move due to school closure. 

Finally, the discussion above focuses specifically on the effect of school closure on student 
outcomes, and ignores the issue of school consolidations. While closing a school is mechanically 
different from a consolidation (where staff from both schools are generally retained and student bodies 
from former schools are merged), one could expect the consolidation effect on students to be similar. 
Under consolidation, learning for all students would likely still sustain some disruption from merging 
disparate school practices and cultures, adapting to new peers and school staff, and likely burdening at 
least some families with a longer commute." Because consolidation of two underutilized schools near 
each other played a prominent role in DCPS’s school-closure strategy, we combine closures with 
consolidations in estimating the effect on student outcomes.” Additionally, because students in 
receiving schools are potentially also affected by the closure and consolidation decision, we include an 
alternate categorization of the treatment in which all students in closed, consolidated, and official 
receiving schools are in the treatment group. We label this categorization “restructured” schools. 
Accordingly, the estimated restructuring effects presented are averaged across the groups. 

This analysis contributes to prior evidence on school closure in two important ways. First, it 


provides an additional rigorous, empirical analysis of the effect of a large school closure initiative, which 


'S School closure also has the potential to influence students in the receiving schools, who may be adversely affected 
with the influx of new students. Engberg et al. (2011) address this issue and find no effect on student achievement 
among students in receiving schools. This issue is beyond the scope of our current analysis. 

' We know of no research that has attempted to analyze the consolidation effect separately from that of closure, 
though several of the analyses on school closures presented above entail other simultaneous interventions in schools 
beyond those that closed. For instance, the Engberg et al. (2011) study coincided with comprehensive reform models 
being implemented in low-performing schools that were selected to stay open in spite of low performance. Also, the 
Chicago study from de la Torre and Gwynne (2009) investigates the closing of 38 schools over a five-year period, 
which dovetailed with the Renaissance 2010 initiative that intended to open over 100 high-quality high schools by 
2010; hence, school closings and new openings were occurring simultaneously for some of the years of analysis. 

'S Estimates on school closures and school consolidations separately were generated, but the standard errors (due to 
the relatively small sample size in both groups alone) were too large to provide meaningful inference. 
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helps to further the research base on this increasingly common policy. Second, we explore student 
mobility in a variety of ways that are plausibly induced as a result of school closure and that have 


received limited attention in the prior research. 


Data 


The District of Columbia Office of the State Superintendent of Education provided access to the 
longitudinal data used in this study. The data contain student records for all students in public schools in 
Washington, D.C., comprised of students in the traditional public school system (i.e., DCPS) and those in 
public charter schools. Student-level information contained in the data includes gender, race and 
ethnicity, eligibility for free- or reduced-price lunch, and special education status, which are used as 
covariates in the analysis. Additionally, student residence is available for two school years (2007-08 and 
2008-09), which is used to identify residential mobility after restructuring and to link neighborhood 
characteristics (e.g. household income, educational attainment, crime rates) that we extract from 
external sources such as the Census and the Metropolitan Police Department and incorporate as 
covariates in the model. 

The data document all student outcomes in tested grades (grades 3-8 and 10 in Washington, 
D.C.) over four school years, spanning 2006-07 through 2009-10. The standardized test used for 
accountability purposes in Washington, D.C. is the D.C. Comprehensive Assessment System (DC CAS), a 
criterion-referenced test that evaluates student learning in reading and math. The test’s raw scale 
scores represent the grade and approximate percentile of the student’s performance (e.g., 542 is a Sth 
grader scoring at what is roughly the 42nd percentile), so the scores themselves are not comparable 
across grades, though comparable over years for the same grade.’° For the purposes of this study, we 


standardize all test outcomes by grade and year. As a result, estimated differences in student 


'© Technical information on the DC CAS is available at http://osse.dc.gov/seo/cwp/view.a,1274.q,561249.asp. 
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achievement are interpreted relative to peer students. These differences do not equate to an absolute 
measure of learning. 

The DCPS school closure and consolidation initiative occurred within the span of the data. In the 
spring of 2008, the final list of schools slated for restructuring was made public. In total 23 campuses 
were closed, 21 of which were elementary and middle schools that conducted end-of-grade tests and 
are included in this study. Of the 21 campus closures, 10 schools were simply closed, and students in 
those schools were reassigned to various neighboring schools. The remaining 11 campuses were closed, 
with students consolidated with a matched set of another 11 schools. This list of affected schools is 
documented in Table 1.”” 

In total, 21 elementary and middle school campuses closed, and a total of 32 schools were 
directly involved with a closure and/or consolidation in the summer before the 2008-09 school year. In 
addition, 19 schools were designated as official receiving schools for dislocated students but were not 
directly closed or consolidated.*® Because of the diversity of the treatments involved in these DCPS 
closures, we cannot reliably separate the effect of any one treatment. Instead, an aggregated effect is 
estimated for all closed or consolidated schools, and then a separate ‘restructuring’ effect is estimated 
for all schools involved in the initiative (comprised of closed, consolidated, and official receiving 
schools). Though the point estimates vary across these two designations, the substance of the principal 
findings is consistent. 

Table 2 presents descriptive statistics for the data sample used for this analysis. Inclusion in the 


sample is limited to students who were in a non-promotional grade in a DCPS school in the 2007-08 


'’ Four of the 11 closed campuses were temporary closings while renovations were completed with the intention that 
the consolidated student body (composed of students from both former schools) would move into the renovated 
building, and the building from which they moved would be permanently closed. As a result, a considerable number 
of students in the non-closed, consolidated schools were affected by the announcement because their building was 
slated for permanent closure in the near future. The column on campus closure notated in Table | is for campuses 
closed during the 2008-09 and 2009-10 school years (the post-closure period in this study), but may not necessarily 
be a permanent status. 

'S DCPS, Office of Public Education Facilities Modernization (Map Schools By Ward), 2009. 
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school year and have valid test scores from that year.’” The columns of table 2 represent four different 
segments of the student population: those who were in schools whose campuses closed in 2008 
(column 1), those in schools that were consolidated (column 2), those in receiving schools (column 3), 
and those not affected in the closure initiative (column 4). The means depicted in this table show 
students in consolidated schools performed significantly higher in the 2007-08 school year (just prior to 
closures) compared with students in closed schools or receiving schools, showing DCPS’ preference for 
favoring the more successful school and closing the weaker when consolidating schools. 

Those students displaced by the closure of their school, who elected to remain in the public 
school system in DC, were presented with the same three options available to all school-age children in 
DC: (1) they could attend their new ‘in-boundary’ public school as determined by their residential 
address and the realigned school catchment boundaries; (2) they could enter the DCPS out-of-boundary 
lottery, in which case they would be eligible to attend an out-of-boundary school of their choice if their 
lottery number was selected; or (3) they could apply to attend a DC charter school, which involves a 
lottery when applicants exceed available slots. Table 2 also lists the observed mobility rate among 
students in the sample for several types of moves, including residential mobility, moving to a charter 
school, or moving out of the DC public school system entirely. As shown in the table, even though 
students in the closed schools were the only ones induced to move, the table shows considerable 


student mobility across all groups, regardless of the school’s restructuring status. 


Methodology 


This study proposes to investigate the relationship between this large-scale school restructuring 
initiative in DCPS and student outcomes, specifically achievement on standardized tests and student 


mobility. Both of these outcomes are addressed in detail below. 


'° Students who were in the terminal grade of a closing school in the 2007-08 school year were to be promoted and 
changing schools regardless of the closure decision, and are therefore omitted from the sample. 
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Student Achievement 

Student learning is a cumulative process in which current achievement levels are a function of 
prior learning (Todd and Wolpin, 2003). To avoid attributing pre-existing differences in student learning 
to school closures, a basic value-added model is used to estimate the restructuring effect using ordinary 
least squares. We depart from the standard value-added literature, however, by using student scores in 
2007 as the prior achievement level for all future test outcomes (spanning 2008 through 2010), rather 
than a once-lagged measurement. We choose to use the 2007 achievement level because this was 
obtained prior to public knowledge of the closure plans, and therefore we treat it as exogenous; a once- 
lagged measure could include an endogenous response of achievement due to closure. In this model, 
student i’s achievement in school s at time t (Aj) is a function of 2007 achievement (Aj2997), a vector of 
student and school characteristics (X;,)”°, an indicator variable for being in a school in the treatment 
group (either closed or consolidated, or restructured) (/treqiment), and a random error term (€;<¢): 

Aist = Bo + Aizoo7P1 + XitB2 + ItreatmentP3 + Eist (1) 

This model will produce consistent estimates of the treatment effect (B3), as long as the 
student-level error term is uncorrelated with current learning, conditional on the included covariates. 
This may be an unrealistically strong assumption in the case of restructuring, since students who are in 
schools that need to be restructured may vary in unobservable ways from otherwise similar students 


who are not in those schools. Importantly, one might expect students who attend these underutilized 


°° Achievement variables include the standardized test scores and proficiency indicators in reading and math in 
Spring 2007. Student characteristics included in the analysis can be categorized into three main groups: 
demographic and socioeconomic; neighborhood; and household characteristics. The first category includes 
race/ethnicity, gender, free or reduced price lunch eligibility, English proficiency, special education, grade and year 
fixed effects; neighborhood characteristics (based on the census tract of residence) include the percent of population 
under 17, percent homeowner, log median income, percent with less than high school degree, with high school 
degree, with less than college degree, and with college degree, average reading and math scores in Spring 2006, 
percent black, Hispanic and FRL eligible; and household characteristics (based on students sharing the same 
residential address and last name) include average prior year reading and math scores, number of school-aged 
children, number of children attending out-of-boundary public or charter schools at the household level. The school 
characteristics included in the models are log enrollment in 2007, percent utilized, percent enrollment change 
between 2002 and 2006, reading and math proficiency rates in 2006, building square footage, number of charter and 
traditional public schools within 3 miles in 2007, and log median income, percent with college degree, percent black 
and Hispanic, property and violent crime rates within the school’s census tract. 
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schools may persist there because their parental involvement is disproportionately low (more involved 
parents in the neighborhood may have already sent their children to alternative public or charter 
schools), and this unobservable attribute hinders student achievement; the restructuring effect is 
downwardly biased in this case. The same argument could be applied for students’ school choice after 
being displaced from their school: disproportionately low levels of parental input in the school choice 
decision may potentially bias the treatment effect downwards, compounding it with the simultaneous 
effect of selection after being treated. 

To compensate for the unobserved endogeneity between school closure and school choice 
before and afterwards, an instrumental variables strategy is used.”* The distance from the student’s 
school in the 2007-08 school year to the nearest ‘relevant’ traditional public school (i.e., serving a similar 
grade range) is used as an instrument. This instrument should be correlated with a school’s likelihood of 
receiving treatment (as it played a significant role in DCPS’ choice of closed, consolidated and receiving 
schools), but is expected to affect student outcomes only through its effect on the restructuring 
decision. 

Because the endogenous treatment variable is binary, we use the two-stage instrumental 
variables strategy outlined in Wooldridge (2002). The first stage predicts the likelinood of receiving 
treatment (l;-eatment, in this case) as a function of the instrument (Z;,-) and all of the other included 
covariates using a linear probability model: 

Pr(Itreatment) = Yo + Zist¥1 + Aizoo7Y2 + Xit¥3 + Fist (2) 

We then use the predicted probabilities obtained from the first-stage as the instrument to 
estimate equation (1) by IV. This two-stage method will produce consistent and efficient estimates of 
the treatment effect. Comparing this method with the typical IV approach (wherein the instrument is 


adopted directly into the IV procedure, rather than being first used as a predictor and using the 


*! Engberg et al. (2011) is the only other school closure analysis that uses an instrumental variables strategy. 
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predicted probability of treatment as the instrument), the estimates under both models are consistent, 
but those resulting from the Wooldridge (2002) method are more efficient. 

In our analysis investigating the relationship between closure and student achievement, we test 
for significant differences in student achievement that arise due to the school restructuring that 
occurred in DCPS. We investigate changes in reading and math achievement that occurred in the year of 
the announcement (i.e., spring testing in 2008) and in the two years following the restructuring (2009 
and 2010). 

Student Mobility 

The school restructuring required many students to move due to school or campus closure, and 
may have also induced additional mobility among the student population. We formally investigate three 
different forms of student mobility: residential mobility manifest in the school year immediately after 
the closures (2008-09); mobility into charter schools over the two years following restructuring (2008-09 
and 2009-10); and mobility out of the DC public school system entirely over the following two years. For 
each type of move, an indicator variable is generated, which is the dependent variable in a linear 
probability model that has the same explanatory covariates as those described in Equation (1): 

Pr Umove,ist) = Bo + Aizo07Pi + XitB2 + ItreatmentP3 + Fist (4) 

And like the student achievement equation above, straightforward estimation of this 
relationship is subject to bias from unobservable differences in students that are non-randomly sorted 
across schools. One could expect students in underutilized, low-performing schools to have a 
systematically higher likelihood of mobility relative to students in schools that are not flagged for 
restructuring, which will bias the estimated coefficient on the treatment upwards in this case. Again, we 
use the two-stage instrumental variables strategy to generate and substitute the estimated probability 
of being in a restructured school into Equation (4) above. The results of this investigation are described 


in detail below. 
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Results 


We begin by first estimating equations 1 and 4 using an ordinary least squares approach. The 
estimates of the treatment effect are presented in Table 3 (each cell represents a separate regression). 
The dependent variables for the regressions are listed as row headings, and all regressions in a single 
row Share the same sample size, only varying in the categorization of which students are receiving 
treatment. Column 1 reports the effect when school closure alone is the treatment of interest; column 2 
represents the combined effect of closure or consolidation; and column 3 represents the general effect 
of being in any restructured school (any closed, consolidated, or receiving school). 

These estimates in Table 3 show mean student achievement in the announcement year and in 
the year following the actual closures were significantly lower among students in closed or consolidated 
schools (column 2) and students in restructured schools (column 3). Comparing the point estimates 
across the columns, one may conclude that the largest apparent decline in student achievement was 
incurred among students in consolidated schools (the point estimates in column 2 are uniformly lower 
than either columns 1 or 3 for all student achievement regressions). The largest estimated effect on 
student mobility appears to arise in the case of movement towards charter schools in the year 
immediately following the closures (both columns 1 and 2 are statistically significant). Other types of 
mobility showed no significant variation. Keep in mind, however, that the estimates reported here may 
be biased due to the selection effect of students across schools. 

To counter the endogeneity of student placement across schools, Woodridge’s (2002) strategy is 
used to consistently and efficiently estimate the treatment effect (a binary variable). The first stage of 
this estimation requires the estimation of the probability of treatment with the chosen instrument and 
other covariates included in the model. Select coefficients on a probit regression (analogous to the first- 
stage linear probability model actually used in the first stage) are reported in Table 4. The coefficients 


reported in this table are based on the sample of students included in the regression with 2009 test 
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score outcomes in reading as the dependent variable (compare the sample sizes from that in Table 3). 
Column 1 predicts the likelihood of being in a closed or consolidated school; column 2 represents 
restructured schools. Coefficients on all other samples were estimated in the first stage of the two-part 
estimation process and were qualitatively similar to those presented in Table 4; however, they are 
omitted here for brevity. 

As shown in Table 4, the point estimates on changes in enrollment were largest in absolute 
value among all of the covariates that are included in the primary model. This is not surprising as this 
measure was one of the two primary criteria for determining whether a school will be considered for 
closure. The other criterion, enrollment in 2006, was also statistically significant, and performance 
metrics for the schools had some additional explanatory power as well. The sole instrument used in this 
analysis, proximity to the nearest relevant traditional public school, was both highly significant and 
predictive, showing the density of schools in a neighborhood as an important predictor of being 
targeted in the restructuring initiative. 

The predicted treatment probabilities are retained and used as the instrument for the binary 
variable of restructuring in the second step. The resulting second-stage IV estimates from this 
specification are presented in Table 5. Each cell in this table represents the coefficient estimate on the 
instrumented restructured variable from a separate regression, with dependent variables listed in the 
table’s row headings. Robust standard errors are clustered at the interacted school and attendance area 
levels, following the methods described in Cameron et al. (2010). 

For the estimated effects on student achievement, these instrumented results are qualitatively 
similar to the ordinary least squares estimates: students showed lower achievement in the year of the 
announcement before the closures took place (Spring 2008) and in the year following the closure 
implementation (Spring 2009). The point estimates for students directly affected by either school 


closure or consolidation (column 1) compared to those in any restructured schools (column 3) are not 
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Statistically different from each other. The magnitude of the estimated effects are quite large; for 
comparison, an effect size of 0.10 student standard deviation units represents approximately 20 to 30 
percent of a full school year’s gains, or two to three months of teaching on a ten-month calendar.” Yet, 
in spite of this large, adverse effect in the short term, the estimates for student achievement in the 
second year following closure (Spring 2010) show no statistically significant difference from zero. 

Many schools involved in the restructuring were among the lowest-scoring in the district. 
Although improving student learning was not the foremost objective in the restructuring process, many 
marginal closure decisions were justified using school performance (preserving relatively stronger 
schools while closing weaker schools) under the expectation that such an approach would result in 
improved performance. Summarizing our results from above, however, students appear to have been 
academically set back temporarily as a result of the necessary school restructuring. Students rebounded 
in year two after closure. These findings are similar to those previously documented in the literature: de 
la Torre and Gwynne (2009) estimate a decrease in achievement of approximately one and a half 
months of learning in the year of the closure announcement, but no additional effect in following years. 
Engberg et al. (2011) report significantly lower achievement (with magnitudes similar to ours) among 
students in the year following school closure but few significant differences beyond that. 

A principal concern with instrumental variable estimation is whether the instruments are strong 
enough to predict the endogenous variable. The instrument appears to be sufficiently strong; Table 5 
reports the F-statistic on the significance of the instrument in predicting whether the school will be 
restructured (columns 2 and 4), and in each regression the F-statistic is large and statistically significant. 

Moving the discussion to student mobility, the estimates reported in Table 5 show no evidence 
that would support that students in affected schools are induced to be more mobile as a result of the 


restructuring. Three different forms of mobility are investigated—residential mobility, mobility to 


* Authors’ calculations using ECLS-K data; results available on request. 
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charter schools, and mobility out of the local public school system entirely—and none of the outcomes 
increase as a result of the closures and restructuring. The estimate on students exiting the public school 
system in DC within two years is significantly lower (at the 5% level) among students in restructured 
schools. This estimate is an anomaly compared to the other point estimates, and in the opposite 
direction of what is generally feared under closure policies (i.e., inducing students to move because of 
the closures). 

On balance, we find no evidence that would support any of these various forms of student 
mobility increasing as a result of the school closure strategy. This result appears inconsistent with the 
evidence presented in de la Torre and Gwynne (2009) that suggests increased subsequent mobility 
among displaced students. However, because they do not address the unobservable propensity for 
students to move (whether displaced students are more likely to move anyway), their estimates are 
likely to be biased upwards, at least as suggested by a comparison of our mobility results obtained with 


OLS (in Table 3) with those from IV (in Table 5). 


Conclusion 


School closures are an occasionally necessary part of managing schools with dynamic student 
populations. In addition, school closure is promoted as a strategy to gain leverage over chronically low- 
performing schools. Yet, few prior studies have been conducted on school closures to understand how 
these actions might influence students. This paper contributes to that literature by evaluating student 
achievement and mobility outcomes in a recent large-scale closure effort in Washington, D.C. Using 
longitudinal data, we investigate how student outcomes change in relationship to this initiative with an 
instrumental variables strategy that counters the endogeneity of student assignment across schools 


before and after the restructuring occurred. 
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, 


The title of this article asks whether students get a ‘leg up’ (improved outcomes) or a ‘boot out 
(increased mobility) as a result of school restructuring. The analysis presented here supports neither of 
these predicted relationships unqualified. The instrumental variables estimates show students in 
affected schools suffered a short-term adverse effect on achievement in both the year of the closure 
announcement and in the year immediately following the implementation of the policy. By the second 
year after closures, however, no significant difference was apparent among affected students. We also 
found no evidence to support the hypothesis that school displacement engendered further student 
mobility. 

Some important limitations apply to these findings. First, improving student performance was 
not a primary objective of the school closures in DCPS, and alternate closures that explicitly seek to 
improve student outcomes by relocating students to high-performing schools may have different results 
than those observed here (Engberg et al., 2011). Second, student outcomes are investigated only along a 
narrow range of measures and we cannot determine the effects on other dimensions that we do not 
observe in the data. For instance, closures could affect students’ relationships with peers and teachers, 
or students’ attitudes towards or engagement in school; these may be important determinants of 
students’ future educational outcomes but unobservable to us in this study. And third, due to data 
available for analysis, we cannot evaluate any closure effects on students outside of the narrow band of 
tested grades, and our findings may not generalize to other grades. In particular, Kirshner et al. (2010) 
suggest a large negative relationship between a high school’s closure and student learning, graduation 
rates, and peer relationships. In short, closure may affect high school students differently than those in 
elementary or middle grades. 

So does closing schools make sense from a policy perspective? We find evidence affected 
students do suffer academically in the very short run (year of announcement and first year in new 


school); however, test scores among students affected by the closures show no difference in the 
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relatively short term (2 years out) from those not affected. In addition, student mobility did not appear 
to increase as a result of the closures, and thus may not impose as large a cost on students as previously 
speculated. On balance, the results of this investigation suggest appropriate caution is warranted in 
choosing whether to close schools, either for policy or practical purposes. None of the evidence from 
school closures in DCPS, however, would indicate that the costs among students are prohibitively large 


so as to avoid closure altogether. 
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Tables and Figures 


Table 1 
Restructured Elementary and Middle Schools in DC in Summer 2008 
% 
Enrollment Closed 
Change Enrollment 
School Name (2002-2006) in 2006 Campus Organization Consolidated Receiving School(s) 
Amidon -35% 227 ° Amidon-Bowen ES 
Backus -67% 141 ° ° LaSalle-Backus 
Benning -25% 162 ° ° Smothers and Plummer 
Bowen -21% 221 ° ° Amidon-Bowen ES 
Brookland -29% 247 ° ° Brookland EC @ Bunker Hill 
Brown, Ronald H -49% 263 ° Ronald H Brown MS 
Browne -27% 211 ° Browne Junior HS 
Bruce-Monroe -14% 307 ° Bruce Monroe ES 
Bunker Hill -42% 270 ° Brookland EC @ Bunker Hill 
Clark -29% 200 . ° Powell and Raymond 
Cook, J.F. -22% 184 ° ° Emery and Walker-Jones 
Eliot -14% 155 ° Eliot-Hine MS 
Francis -21% 165 ° Francis-Stevens EC 
Gage-Eckington -35% 183 ° ° Emery, Seaton, Shaed, and Garrison 
Garnet-Patterson -19% 206 ° Shaw MS @ Garnet Patterson 
Gibbs -44% 281 ° ° Browne Junior HS 
Green -35% 238 ° Turner ES @ Green 
Harris, Patricia R. -31% 631 ° ° Patterson, Leckie, and Hendley 
Hine -44% 268 ° ° Eliot-Hine MS 
Merritt -48% 170 ° ° Ronald H Brown MS 
Meyer -49% 169 ° ° Tubman, Garrison and HD Cooke 
Moten -32% 215 ° Moten ES @ Wilkinson 
Park View -45% 162 ° ° ° Bruce Monroe ES 
Rudolph -50% 228 ° ° Truesdell and Whittier 
Shaw -38% 154 ° ° Shaw MS @ Garnet Patterson 
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(Table 1 continued) 


Slowe -64% 83 . ° Burroughs and Noyes 
Stevens -28% 231 ° ° ° Francis-Stevens EC 
Turner -25% 318 ° ° Turner ES @ Green 
Webb -42% 327 ° ° Webb-Wheatley ES 
Wheatley -80% 48 ° Webb-Wheatley ES 
Wilkinson -28% 365 ° ° Moten ES @ Wilkinson 
Young -29% 243 ° ° Browne Junior HS 


Notes: Authors’ compilation from district data. 
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Table 2 
Descriptive Statistics 


Student’s school in 2007 was... 


Consolidated A 
but not receiving Not 
Closed closed school altered 
Standardized reading score in Spring 2007 -0.285 -0.209 -0.410 -0.019 
(0.863) (0.926) (0.966) (1.051) 
Standardized math score in Spring 2007 -0.423 -0.317 -0.514 -0.128 
(0.825) (0.975) (0.946) (1.045) 
Standardized reading score in Spring 2008 -0.244 -0.144 -0.331 0.073 
(0.861) (0.956) (0.948) (1.058) 
Standardized math score in Spring 2008 -0.307 -0.142 -0.403 0.017 
(0.841) (0.986) (0.952) (1.06) 
Standardized reading score in Spring 2009 -0.246 -0.080 -0.298 0.144 
(0.896) (0.954) (0.966) (1.057) 
Standardized math score in Spring 2009 -0.263 -0.102 -0.300 0.146 
(0.908) (0.979) (0.949) (1.073) 
Standardized reading score in Spring 2010 -0.251 -0.147 -0.278 0.076 
(0.951) (1.051) (1.043) (1.141) 
Standardized math score in Spring 2010 -0.194 -0.039 -0.183 0.161 
(0.906) (0.93) (1.002) (1.071) 
Changed residences after restructuring 0.158 0.180 0.170 0.132 
(0.365) (0.384) (0.376) (0.338) 
Attending another school in 2008-09 school year —_1.000 0.239 0.343 0.240 
(0.299) (0.426) (0.475) (0.427) 
Moved to a charter school in 2008-09 school 
year 0.225 0.120 0.167 0.112 
(0.418) (0.325) (0.373) (0.316) 
Moved to a charter school within two years 0.304 0.245 0.257 0.231 
(0.46) (0.43) (0.437) (0.421) 
Left DC public school system in the year after 0.099 0.090 0.091 0.132 
(0.299) (0.286) (0.288) (0.339) 
Left DC public school system within 2 years 0.229 0.224 0.264 0.265 
(0.421) (0.417) (0.441) (0.441) 
Black 0.954 0.889 0.911 0.759 
(0.209) (0.314) (0.285) (0.428) 
Free or reduced priced lunch eligible 0.796 0.775 0.798 0.640 
(0.403) (0.417) (0.402) (0.48) 
Residential census tract characteristics: 
Median household income 25914.5 27189.9 26308.5 34225.9 
(7103.7) (7568.5) (8137.1) (18882.2) 
Percent with college degree 0.104 0.117 0.105 0.148 
(0.064) (0.065) (0.074) (0.101) 
N 789 1,539 1,404 4,661 


Notes: Standard deviations are given in parentheses. Number of observations in each category includes all 
students in grades 4-8 who were in a non-promotional grade at a DCPS school in 2007. 
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Table 3 
Effects of School Restructuring on Achievement 
OLS Results 


Student’s school in 2007 was... 
Consolidate Restructure 


Dependent Variables: Achievement Closed d or closed d N 
Standardized reading score in 2007 -0.037 -0.057 -0.044 7,198 
(0.058) (0.035) (0.032) 
Standardized math score in 2007 -0.010 -0.088* -0.037 7,191 
(0.058) (0.052) (0.049) 
Standardized reading score in 2008 -0.063 -0.102** -0.081** 6,461 
(0.053) (0.046) (0.039) 
Standardized math score in 2008 -0.086 -0.132** -0.097** 6,456 
(0.054) (0.055) (0.047) 
Standardized reading score in 2009 0.015 -0.017 0.024 4,412 
(0.056) (0.047) (0.041) 
Standardized math score in 2009 0.013 -0.028 -0.016 4,410 
(0.065) (0.058) (0.053) 
Dependent Variables: Mobility 
Changed residences after restructuring -0.009 0.008 0.016 6,764 
(0.015) (0.013) (0.010) 
Moved to a charter school in 2008-09 school 0.042* 0.037** 0.005 6,764 
year 
(0.024) (0.015) (0.011) 
Moved to a charter school within two years 0.037 0.009 -0.017 6,007 
(0.031) (0.027) (0.019) 
Left DC public school system in the year after -0.011 0.003 0.024 7,206 
(0.026) (0.016) (0.017) 
Left DC public school system within 2 years -0.063 0.001 0.008 7,206 
(0.034) (0.023) (0.022) 


Notes: For each regression, test scores are standardized to mean zero and unit variance. Robust standard errors, two-way 
clustered at the school and attendance area levels as described in Cameron et al. (2010), are given in the parentheses. All 
regressions include the covariates listed in footnote 15 of the text. *, ** and *** represent statistical significance at 10, 5 
and 1| percent respectively. 
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Table 4 
Determinants of School Restructuring 


Probit Results — Marginal Effects 


Dependent variable 


Consolidated 
or closed Restructured 
% enrollment change 02-06 -0.394*** -1.049*** 
(0.088) (0.345) 
Log (Enrollment in 2007) -0.149*** 0.858*** 
(0.053) (0.272) 
Reading proficiency rate in Spring 2006 -0.601*** -3.034*** 
(0.185) (0.569) 
Math proficiency rate in Spring 2006 0.140 1.306*** 
(0.142) (0.546) 
Distance to the closest ‘relevant’ traditional public -0.322*** ~L,915*** 
school 
(0.080) (0.282) 
N 6,461 6,461 


Notes: The estimates present the first-stage probit results (in marginal effects) of the two-stage procedure described in 
Wooldridge (2002) where the sample is the students in grades 4 through 8 with non-missing test scores in Spring 2008 
who were in a non-promotional grade at a DCPS school in 2007. In addition to those listed, both regressions 
include the covariates listed in footnote 15 of the text. Robust standard errors, clustered at the school level are given in 
the parentheses. *, ** and *** represent statistical significance at 10, 5 and 1 percent respectively. 
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Table 5 
Impact of School Restructuring on Achievement 


IV Estimates 
Student’s school in 2007 was... 
F-stat of F-stat of 
joint joint 
significance significance 
Consolidated (excluded (excluded 

Dependent Variables: Achievement or closed instrument) Restructured instrument) 
Standardized reading score in Spring -0.111** 18.77 -0.068 22.96 
2008 

(0.041) (0.047) 
Standardized math score in Spring 2008 -0.199*** 18.82 -0.120** 23.02 

(0.050) (0.051) 
Standardized reading score in Spring -0.102** 18.25 -0.115** 22,59 
2009 

(0.051) (0.054) 
Standardized math score in Spring 2009 -0.120* 18.25 -0.125* 22.58 

(0.068) (0.076) 
Standardized reading score in Spring -0.023 11.23 0.034 25.95 
2010 

(0.061) (0.057) 
Standardized math score in Spring 2010 0.140 11.26 0.096 25.88 

(0.102) (0.089) 
Dependent Variables: Mobility 
Changed residences after restructuring 0.017 18.23 0.013 22.56 

(0.017) (0.018) 
Moved to a charter school in 2008-09 0.028 18.56 0.021 23.04 
school year (0.020) (0.020) 
Moved to a charter school within two -0.038 18.30 -0.001 22.10 
years 

(0.033) (0.031) 
Left DC public school system in the -0.003 18.78 0.004 22.92, 
year after (0.012) (0.009) 
Left DC public school system within 2 -0.064*** 18.78 -0.061** 22:92 
years (0.021) (0.025) 


Notes: For each regression, test scores are standardized to mean zero and unit variance. Robust standard errors, two-way 
clustered at the school and attendance area levels as described in Cameron et al. (2010), are given in the parentheses. All 
regressions include the covariates listed in footnote 15 of the text. *, ** and *** represent statistical significance at 10, 5 
and 1| percent respectively. 
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